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Abstract 

Wide-angle images gained a huge popularity in the last years due to 
the development of computational photography and imaging technological 
advances. They present the information of a scene in a way which is more 
natural for the human eye but, on the other hand, they introduce artifacts 
such as bent lines. These artifacts become more and more unnatural as 
the held of view increases. 

In this work, we present a technique aimed to improve the perceptual 
quality of panorama visualization. The main ingredients of our approach 
are, on one hand, considering the viewing sphere as a Riemann sphere, 
what makes natural the application of Mobius (complex) transformations 
to the input image, and, on the other hand, a projection scheme which 
changes in function of the held of view used. 

We also introduce an implementation of our method, compare it against 
images produced with other methods and show that the transformations 
can be done in real-time, which makes our technique very appealing for 
new settings, as well as for existing interactive panorama applications. 

Keywords: panorama, perspective projection, Mobius transformation, 
real-time, implementation. 


1 Introduction 

An image is called panoramic when it represents a wide held of view (FOV, in 
photography, is the part of the world which is visible through the camera). 

Some applications generate panoramic images from an image representing 
the viewing sphere^ a sphere containing the scene, centered at the viewpoint. 
The viewing sphere is usually obtained with special cameras, software or a 
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combination of both. A panorama can be obtained by stitching together a set 
of regular pictures of a scene, producing an image inscribed on the surface of 
a sphere, and then the sphere projected on a plane. The entire process can 
be done by using programs like Hugin [6]. This software uses state-of-the-art 
methods to find common points in different images of the same scene, combine 
those images, correct colors and project the sphere. In the last years, however, 
cameras capable of directly obtaining a 360-degree sphere became popular; these 
cameras directly output a spherical image, without the need of special software 
to combine pictures m- There also exists the well-known fisheye lens [10], 
which permits to capture a 180-degree image directly on a plane. 

There exists a number of transformation techniques to obtain a panoramic 
image from the viewing sphere (typically, a projection from the sphere to a 
plane). Each type of transformation has distinct properties, and the goal of 
using different transformations is usually to obtain a more realistic image. The 
subjective concept of realistic can be interpreted in different ways, typically as 
bending straight lines as less as possible or as preserving angles of the scene. 
Since lines and angles cannot be preserved at the same time m, different pro¬ 
jection schemes were proposed. Warping techniques were also developed [SllISI; 
however, they need long human interaction or optimization methods and thus 
cannot be used for real-time nor interactive applications. 

While one kind of transformation might be good for some setting, it might 
be bad for other. This situation becomes evident on some interfaces where a 
user is able to interactively change the FOV of a scene. Usually, perspective or 
equiroctangular projections (both known for providing good results for small to 
medium FOVs) are used for interactive visualizations. When the FOV becomes 
very wide, projections are perceptually less realist. 

What we propose in this paper is to adopt a new approach in interactive ap¬ 
plications for obtaining a panoramic image when the FOV becomes wide. When 
the user widens the FOV until surpassing a limit where common projections 
tend to produce bad results, we propose to simulate the widening of the FOV 
by performing a Mobius transformation, and then apply the initially intended 
common projection. The novelty of this approach is that we do not introduce 
a complicated projection method but, instead, we perform a transformation on 
the viewing sphere and then we project using a well-known perspective projec¬ 
tion. Performing transformations of the image directly on the viewing sphere is 
specially important in some settings, on which it is not desired to project on a 
plane (for instance, when projecting on a dome) and, as far as we know, is also 
a novelty of our method. 

Our main contribution is a projection method which performs a transforma¬ 
tion on the viewing sphere, which makes it applicable on non-plane projections. 
Moreover, our method can be used in real-time applications. We also use a pow¬ 
erful tool such as complex transformations for our purpose, showing that they 
can be employed in Image Processing and open the quest for new applications 
in the field. Finally, we present an open-source implementation of the method. 

A preliminary version of this work was published in m- The present full- 
version includes the theoretical foundations of the method, new experiments 
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(comprising comparison with more state-of-the-art methods), argues about per¬ 
ceptual properties of the generated images, adds a discussion on the applications 
of the new technique and motivates future research. Additionally, the text was 
entirely rewritten. 

The roadmap of the paper is as follows. Next Section argues about previ¬ 
ous work on the field. Section |3] introduces some mathematical definitions to 
understand the rest of the paper. Our approach is formalized in Section]^ Sec¬ 
tions and provide an analysis of the technique, while Section [7 introduces 
some scenarios where our technique can be applied. Finally, Section ^discusses 
current work and future research directions. 


2 Previous Work 

Due to their increasing popularity and interest, panoramic images have become 
a theme of intense discussion in the Computer Graphics and Image Processing 
communities in the last twenty years. The impossibility of obtaining a global 
projection from the sphere to the plane that preserves all possible straight lines 
and object shapes was shown in the seminal work by Zorin and Barr [25]. This 
theoretical limitation motivated much research for obtaining perceptually real¬ 
istic panoramas. 

One approach for this problem was to use different perspective projections 
in the same scene M- The user specifies different projection planes and view 
directions to define the different projections. The discontinuities caused by 
using different projections for different regions of the panorama were hidden (if 
possible) by choosing the projection planes in a way that fit well orientation 
discontinuities that were already present in the scene. 

Other approach consisted in investigating near-perspective projections used 
by ancient painters, such as Pannini projection m- This technique produces 
very good results, as proved in some ancient paintings. 

Conformal mappings were also explored with the aim of preserving the shape 
of the objects in a panoramic image [8]. This approach consisted in investigating 
the stereographic projection and scaling of the complex plane, but only for 
artistic and exploratory purposes. Since the focus was on shape preservation, 
results present bent lines. In our work, we go beyond and improve these ideas 
to map wide fields of view to narrower ones and add a perspective re-projection 
step, which produces better quality results because the straight lines are less 
bent. 

Other methods relied on both user interaction and energy-minimization for¬ 
mulations. Carroll et al [5] used the important lines in the scene provided by 
the user and detected faces to control straight line preservation and conformal¬ 
ity in these regions. Kopf et al. m used regions specified by the user where 
the projection should be nearly planar to formulate their optimization frame¬ 
work. Wei et al. [23] formulates the problem as the minimization of a quadratic 
energy based on user annotation, which has a closed form and implies in the 
solution of a sparse linear system. In these approaches, user interaction was 
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Figure 1: Correspondence between a point p on the Riemann sphere and a point 
p on the complex plane. 


usually laborious and the optimization formulations made them impossible to 
be implemented in real-tim^ 

An interesting approach consisted in dynamically changing the projection 
used depending on the FOV [12]. This can also be achieved by our viewer 
by applying different shrink values as the FOV of the perspective projection 
changes, as explained in Section Also, our visualization simulates better 
camera movements since it is a natural generalization of the perspective projec¬ 
tion. 

Another advantage of our method compared to previous ones is that we 
do not rely on heavy user interaction. The user is only asked to vary one pa¬ 
rameter (which may be set automatically to a good value), what makes our 
panorama viewer a pleasant experience, instead of laborious. Also, our formu¬ 
lation is simple and does not rely on heavy optimizations, which makes possible 
to implement our method in real-time. 

3 Definitions 

Before describing our method, we formalize in this Section the ideas presented 
so far. We assume the given image is inscribed in the viewing sphere. The 
viewing sphere will be considered a Riemann sphere. Each point on the sphere 
corresponds thus to the representation of a complex number; Figure shows 
the correspondence between points in the complex plane and on the Riemann 
sphere. A complex number p = x iy is a point on the complex plane, with 
coordinates x and y. This plane is the stereographic projection of the Riemann 
sphere. The point p in the complex plane is represented on the sphere by the 

^For us, real-time means that the computations are done quicker than the rendering of 
a frame in the application. Even if the other methods are fast, they are not suitable to be 
implemented in real-time. 
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Figure 2: The viewing sphere. The FOV (blue region) is to be mapped to a 
rectangular image, o is the observers position, in the center of the sphere. 


point p. p and p are related by stereographic equations. A complete study 
of stereographic projections is beyond the scope of the present work. We will 
only show the formulas of stereographic projections here and refer to Snyder’s 
book m for details. 

The stereographic projection maps complex points in points on the viewing 
sphere. It is defined as follows. 
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On the other hand, the inverse stereographic projections maps points on the 
viewing sphere to complex points, and is defined as follows. 
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We assume the observer stands in the center of the viewing sphere. She can 
do two basic things: rotate her head in any direction or change the FOV. For 
the sake of simplicity they will be studied separately, but it would not require 
much extra work to consider them together. 

The rotation of the observer’s head is represented as two angles, a (azimuth) 
and A (altitude). These angles represent uniquely a point on the sphere, which 
is associated with the center of the produced image, see Figure [3a| 

The FOV is again represented as two angles, (pa (azimuthal FOV) and (px 
(altitudinal FOV), as depicted in Figure 3b However, the ratio between these 


two angles is defined by the aspect ratio of the output image. 

The problem consists then in obtaining a plane image of the FOV (the part 
shaded in blue in Figure]^ in function of pa and px. 

The parameters a, A and pa {px can be considered a function of pa) deter¬ 
mine the points on the sphere to be processed. Recall that each point on the 
sphere is uniquely determined by px and pa (because the radius of the sphere 
is fixed) or by its cartesian coordinates in the original plane. 
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(a) 6c is the eyesight direction, a is the 
angle between vector 6c and plane yz. A is 
the angle between vector 6c and plane xy. 




Figure 3: Details of the viewing sphere and the view region. 


To study the variation of we assume a and A fixed to zero. This way, 
a value of (pa uniquely determines the four points on the sphere which are the 
vertices of the blue region of Figurej^ For a given pa^ the FOV is computed, and 
points on the sphere are mapped to the output image by a projection function 
(which usually depends on the FOV). 

Changes in viewing directions are given by changing a and A. Since the 
FOV does not change, these changes can be modeled as rotations of the sphere. 
This approach has the advantage that the projection function does not need to 
be recomputed, although it must be composed with a rotation function on the 
sphere. 

Finally, let us mention that the perspective projection is a key ingredient of 
our method. Since it is a well-known projection method, we will not describe 
it, but only mention that it is defined by the following equation. 

P: S2\p<0} ^ C 

p{x,y,z) ^ P 6,^) 

In practice. Equation is scaled, in order for a given FOV to be projected 
in the extents of an image. In the sequel, we will denote a function performing 
a perspective projection of a FOV 0 as P^. We refer the reader to the book by 
Snyder [20] for more details and a complete study of perspective projections. 

4 The Mobius Projection Method 

This Section introduces our method, aimed to visualize images with wide FOVs. 
Perspective projections work very well in practice for small to medium FOVs. 
When (pa grows to a big value, it becomes difficult to conceive a good projection 
function. One approach is to replace the perspective projection by a warp¬ 
ing which respects visual restrictions, such as conformality and preservation of 
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Figure 4: The Mobius transformation, shrinking points in the new FOV (in 
blue) to the old FOV (in red). 


straight lines, as much as possible. We refer to [16] for details on such methods. 
The drawback of these methods is that they are slow and they usually need 
human interaction, what makes them unsuitable for interactive applications. 

Our approach consists in using two different projection methods, depending 
on the desired FOV. To the best of our knowledge, the only work on which 
the projection scheme used varies as a function of the FOV is the paper by 
Kopf et al. m- They use an adaptive projection resulting from an interpolation 
between a perspective and a cylindrical projection. 

We assume that there exists a value ^max, such that the perspective projec¬ 
tion does not introduce big distortions when (pa < 0max- When this happens, 
we propose to use a direct perspective projection, that is, without performing 
any shrink. 

On the other hand, when pa > ^max, the perspective projection fails to 
provide a realistic image. We propose, in this case, to perform a hyperbolic 
Mobius transformation m on the sphere points, in order to map points of the 
new FOV region to the old FOV region, as depicted in Figure]^ (this is achieved 
by first mapping the sphere points to the complex plane, then applying the 
Mobius transformation to the complex plane and, finally, lifting the points back 
to the sphere). After, the same projection maps points on the sphere to 

the output image. 

Our projection schema is non-conformal and does not preserve areas (it in¬ 
herits these properties from the perspective projection). In addition, since hy¬ 
perbolic Mobius transformations do not preserve great circles on the Riemann 
sphere, our method looses the line-preservation property from perspective pro¬ 
jections. We will show in Section that, despite these properties, our method 
produces indeed very realistic images. 

4.1 Computing the Shrink 

As mentioned earlier, the problem is to find the Mobius transformation that 
shrinks the new FOV into the old FOV, as depicted in Figure]^ If we had the 
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Figure 5: Pipeline of the shrinking process. Recall that tghrink = S ^ o Ms o S. 

point c of Figure coinciding with one of the poles, we could simply apply 
an hyperbolic transformation to shrink the FOV. This can be accomplished by 
rotating the sphere and making c coincide with the south pole (corresponding to 
the origin on the complex plane). Mathematically, this means to do a translation 
by a vector ~^) polar coordinates (note that this translation must 

not be done in the equirectangular image). Without loss of generality, we assume 
in the rest of the paper that a = A = 0. 

The following step is to perform a shrink centered in the origin. This can be 
accomplished by using a hyperbolic Mobius transformation. The only knowledge 
about Mobius transformations needed to understand the method is that the 
shrink transformation we need takes the form Mg = pz. We refer the reader to 
Appendix for some more details on Mobius transformations. 

At this point, after being introduced to Mobius transformations, the reader 
should ask why we emphasize that our method uses them, while it is only a 
polar scaling. The reason is that we believe this kind of transformations has a 
myriad of applications in Image Processing, and we want to encourage further 
research on them. 

It remains to compute p. Since 0max and (j) are angles, it is direct to compute, 
in the equirectangular domain, p = But Mq is defined for the points on 

the complex plane: this means that a stereographic projection S must be first 
applied to the points on the sphere, then the old FOV is mapped to the new 
FOV using Ms and, finally, an inverse stereographic projection S~^ is used to 
map back points to the sphere. This means that the shrink transformation takes 
the form tghrink = S~^ o Ms o S. (Note that, if a and A are not assumed to 
be zero, the translation trot must be applied before tghrink, and the the inverse 
translation —trot must be applied after.) 

Finally, the desired projection is obtained by applying the transformation 
P^max resulting shrink. The pipeline of the process is shown in Figure]^ 

The shrink is a good approach to reach wide FOVs on panoramas. However, 
even with the shrink/projection scheme, a FOV close to 27r will produce an 
unrealistic output image, as well as introducing unacceptable distortions. It 
becomes interesting thus to determine the biggest (j) that can be visualized with 
this technique, as well as computing a good value for 0max- The first of these 
shall be determined by experimentation, while the second can be approximated 
by a function of 0, as well as determined experimentally. The next Section 
addresses the computation of (/)max- 












5 A Perceptually-Good Value of (f)niax 


In this Section, we will argue about the value of the parameter 0max- As men¬ 
tioned earlier, a value of 0max can be determined experimentally, given a value 
of (j). 

We will show first that our Mobius shrinking scheme produces the same 
distortion than the stereographic projection by setting 0max = e, a small value. 
As explained in Section |4.1[ given the image inscribed in the viewing sphere, 
our method M can be seen as a composition of functions (we will assume again, 
without loss of generality, that the sphere does not need to be rotated, in order 
to simplify the explanation). 

M = Pe O tshrink 

= Pe O S-^oMsOS 

^ S O S-^oMsoS (4) 

MsoS 

S 

The approximation (in the third line of the equation) is due to the fact that 
e is very small and, in this case, the perspective projection P^ is almost equal to 
the stereographic projection S. The last equality is forced by the fact that MgoS 
means to apply the stereographic projection to the sphere and then shrink the 
resulting image; since this shrink does not deform the image, we conclude that, 
in this setting. Mg has no effect. This result will be corroborated experimentally 
in Section [U 

Preservation of lengths and angles is important for many applications such 
as cartography, but the stereographic projection is not ideal for panorama vi¬ 
sualization since it does not take into account perceptual aspects. In the sequel 
we propose a choice of 0max that approximates better human perception of the 
world. 

It is known that the way images are formed on the human retina is very 
similar to a perspective projection. Furthermore, our eyes only see with clarity 
objects that are in the region called central FOV, which comprises 60 degrees or 
less Hug. We use this fact to propose the value (/)max = 00°. Using such value 
will map points on the portion of the viewing sphere to be visualized to a 60 
degrees FOV and apply a perspective projection to this limited FOV, simulating 
then what one would see if was immersed in the Mobius transformed sphere. 

5.1 Discussion 

We stress that the choice 0max = 60° does not guarantee straight lines in the 
scene to be straight in the final result. This is due to the fact that the Mobius 
transformation on the sphere does not map segments of great circles (projection 
of straight lines on the sphere) to segments of great circles. If the user wants 
straight lines to be better preserved, it is still possible to set a higher value to 
0max- On the other hand, diminishing this value results in a better preservation 
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of lengths and angles. This aspect is further discussed in the Section and in 
the accompanying video. 

Since lines are not preserved, it is of interest to question if there are some 
lines which are preserved, and the perceptual implications of the answer. To 
begin this study, let us clarify how pieces of straight lines (segments) on the 
scene are represented on the viewing sphere. One segment on the scene and the 
center of the sphere form a triangle. The intersection of two sides of the triangle 
with the sphere determines two points. The representation of the segment on 
the viewing sphere is the geodesic joining these two intersection points. The 
triangle is contained on a plane, and the intersection of this plane with the 
viewing sphere is a great circle. In general, straight lines on the scene are 
represented on the viewing sphere as great circles or arcs of great circles. 

Under a hyperbolic Mobius transformation, some circles on the sphere are 
preserved, and some not. The great circles passing by the north pol^ remain 
unchanged under hyperbolic Mobius transformations. The circles parallel to 
the Equator are mapped to different circles parallel to the Equator. The rest of 
the circles lose their shape when applying hyperbolic Mobius transformations. 
This analysis permits to see that, in our method, straight lines are deformed at 
different extents depending on the distance of their projections on the viewing 
sphere to the shrinking point. 

It remains to discuss about how bending lines affect perception. It is widely 
accepted that a projection is better when it bends lines less, but let us postulate 
that this is not quite true in human perception. It is known that the human 
vision is very complex, far beyond the scope of this paper, and that the brain 
reconstructs the image captured by the eyes to form a three-dimensional model 
of the reality. Historically, camera lenses were manufactured to try to produce 
perceptually-good images. One meaningful example on this are the fisheye lenses 
for very wide EOVs, in which lines are bent. We believe in this respect that 
the line bending of our method is perceptually-acceptable for very wide EOVs. 
This fact will also be corroborated in the next Section. 

Let us mention, to conclude the Section, that one approach to partially re¬ 
duce line bending is to consider cylindrical projections. Cylindrical projections 
do not bend vertical lines, but the problem remains intact for horizontal lines. 
A direction to explore in the future would be to combine Mobius transforma¬ 
tions with cylindrical projections, as Kopf et al m do with perspective and 
cylindrical projections. 

5.2 A Final Note on Distortion 

It would be interesting to also perform the analysis from an orthogonal point 
of view, that is, a completely automated method which does not require human 
interaction. As far as we know, the only method aimed to quantify the quality 

^Recall that, in our method, the viewing sphere is rotated to make coincide the north 
pole with the center of stretching, and rotated back after the stretching. For simplicity, we 
will assume here that the shrinking occurs with the north pole as center, without considering 
rotations. 
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Table 1: Frame rates obtained by our technique while the user interacts, 
for different mesh resolutions (varying on the rows) and different input equi- 
rectangular image resolutions (varying on the columns). These results were 
generated in a screen resolution of 1024 x 768 pixels in a PC with an Intel 
Xeon Quad Core 2.13GHz with 12 GB of RAM and a GeForce GTX 470 GPU. 



4000 X 2000 pixels 

8000 X 4000 pixels 

200 X 200 vertices 

93 fps 

89 fps 

400 X 400 vertices 

85 fps 

84 fps 

800 X 800 vertices 

35 fps 

33 fps 


of a plane projection of a sphere is due to Milnor m- Briefly, this methodology 
computes the scale of each pair of points on a projection (the scale is defined as 
the ratio between the geodesic distance on the sphere and the projected distance 
of them). The distortion is the logarithm of the ratio between the smallest and 
the biggest scales of the projection. 

We used Sage Math [21] to implement the formulas of the Milnor distortion. 
For our experiments, we used a set of points on a grid of 500 x 500 points on 
the surface of the unit sphere. We repeated, for FOVs between 1° and 180°, 
the following experiment. We considered all the pairs of distinct points of this 
set (inside our FOV). We computed the projections (using many different pro¬ 
jection methods, including ours) of each point and, for each pair of points, we 
computed the scale under each projection method. This procedure let us com¬ 
pute the Milnor distortion, for a given FOV, for all different types of projections. 
We compared then the distortions for each FOV. The results were not the ex¬ 
pected: the quantitative Milnor distortion of the images does not correspond 
to their perceptual quality. This fact opens a research direction for the future: 
to conceive quantitative methods for projections, consistent with the human 
perception. 

One alternative to our experiment is to choose the set of points to compute 
Milnor distortion on by studying the content of images. However, the results 
will be associated with a specific image and will not be as general as we wanted 
when designed our experiment. We believe it deserves a full research paper to 
study quantitative distortions of specific images. 


6 Implementation and Experiments 

While in theory our method shows advantages over direct projections, we im¬ 
plemented it to see how it performs in practice and the perceptual quality of 
the produced images. We coded in G+-h, using OpenGL to deal with image op¬ 
erations and texture the surface of the viewing sphere, and Qt for the interface, 
what asserts portability. All geometric transformations on the vertices of the 
sphere can be performed in parallel, what made us implement them as a GLSL 
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Algorithm 1: Mobius projection method 

Input : coordinates (x,^, z) of a point in a plane image, angles A and 0 
and a scale s to scale the complex plane 
Output: coordinates {xt^yt^Zt) of the point, transformed using the 
Mobius projection method 

// rotation of —A on the xz plane 

(x, y^z) cos(—A)x — sin(—A) 2 :, y, sin(—A)x + cos(—A)^:); 

// rotation of on the xz plane 

(x, y, z) •(- (x, cos{-4>)y - sin{-4>)z, sm{-(j))y + cos(-^) 2 :); 

// Stereographic projection 
{u,v) ^ {2x/{l - z),2y/{l - z)); 

II convert from Cartesian to polar coordinates 

(p, 6>) ^ arctan(i4, i;)); 

// scaling the complex plane according to s 

p ^ ps; 

II mapping back from polar to Cartesian coordinates 

(rt,'T’) ^ — psin(^),pcos(^)); 

// mapping back from the complex plane to the unit sphere 

{x,y,z) ^ (^4:U/{u^ ^ v‘^4), 4:V/{u‘^v‘^4), {u‘^ ^ v‘^ — 4)/{u‘^v‘^4)); 

II perspective projection 

{xt,yt,zt) ^ {x/{-z),y/{-z),z)] 
return {xt,yt,zt); 


vertex shader. 

The input of the shader algorithm is the viewing sphere, with the equirect- 
angular image as a texture. Each one of the vertices forming the triangular 
grid approximating the sphere is processed by the vertex shader, computing its 
position on the projection plane (applying the operations described in Figure]^ 
on Page[^. Finally, the fragment shader then interpolates the texture values 
inside the resulting triangles, forming the output image. Algorithmic shows the 
pseudocode of the algorithm we implemented in the vertex shader. It should 
be noted that the shader works with homogeneous points but we used, in the 
pseudocode, three-dimensional points, since the homogeneous component al¬ 
ways equals one in our case. In addition, we can safely assume the ^-coordinate 
of the input points to be zero. 

The implementation consists in a window showing the image processed through 
our technique. The interface can be appreciated in the video accompanying this 
paper. It permits to pan the image with the mouse, as well as adjusting the 
FOV and the parameter 0max with the mouse wheel (the latter holding the 
Shift key). All these operations are done in real-time on a desktop PC, as 
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(a) Stereographic projection (b) Mobius, 0max = 1° 0.02rad 

Figure 6: The stereographic projection and the Mobius method with a small 
value of (/)max produce the same results, as predicted in Equationon Section 
In this case, (j) = 240° « 4.2rad. 


illustrated in Table on Page EH The code is open-source and available at 
http://git.impa.br/luisp/panoramic/, 

We reproduce here some results of our tests. First of all, let us corroborate 
the affirmation of Section which stated that our method produces similar 
images than the stereographic projection. In Figure the Roman Coliseum 
serves us to show that the stereographic projection and our method produced 
images which are indistinguishable for the human eye. This figure was produced 
using a very wide FOV. 

Figure shows how our method improves the quality of the images near the 
borders with respect to the perspective projection, while keeping the image on 
the center undistorted. From this figure, as well as from the video accompanying 
the paper, one can realize a meaning of the parameter In the figure, 

line bending is easily distinguishable, since the scene is full of straight lines. 
Moreover, it is evident how lines passing by the center of shrinking are preserved. 

A big value of 0max (bigger than or equal to the FOV) produces a perspec¬ 
tive projection. Small values bend the lines on the scene, but correct artifacts 
of the perspective projection. Smaller values bend lines more and correct more 
artifacts. From the perceptual point of view, bent lines are by far more accept¬ 
able than distorted images, as can be appreciated in Figure We can see 0max 
as a parameter which controls the ratio between line straightness and distortion 
near the borders. The price paid for obtaining images which are clear close to 
the borders is, in this case, loosing the preservation of straight lines. 

We also compare our technique to previous methods in the field of generation 
of panoramic images. We use first a view of the Rheingauer Dom to show in 
Figure a comparison with standard projections from the viewing sphere to 
the image plane. We observe that for the field of view of 160 degrees the 
perspective projection (Figure |^) distorts objects too much but preserves all 
straight lines. The stereographic projection, equal to the Mobius method when 
0max = 1°, (Figurej^) improves the perceptual quality of the image by bending 
some lines. The Mercator projection (Figure]^) preserves object shapes because 
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(a) Perspective 


(b) Stereographic Mobius, 0max = 1° 
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(c) Mobius, 0max = 60° (d) Mobius, 0max = 90° 

Figure 7: It is shown how the parameter (/)max controls line bending. In all 
subfigures, 0 = 172° ^ 3rad. 


it is conformal, but bends much the lines. The perspective projection corrected 
with the Zorin-Barr transformation, using parameters A = 0.2 and R = 0.5 
(Figure 1^), yields an image with a reasonable straight line preservation, but 
presents artifacts near the borders. The Mobius projection using the value of 
0max = 60° (Figure]^) produced a perceptually good result. Finally, we set the 
parameter 0max to 120° (Figure]^) to obtain an image with less line bending. 

In Figure 1^ we compare our method with recent works on the same topic with 
a view of the courtyard of the Hamburg town hall. The result obtained by the 
technique proposed by Zelnik-Manor et al. [24] (Figure]^) shows discontinuities 
on the floor produced by using different projections for different areas of the 
image. This strategy works successfully for the building in the image, since these 
discontinuities are hidden by natural orientation discontinuities in the scene, but 
it fails to fit the geometry of the floor. In Figure we show a result produced 
by our implementation of the technique by Carroll et al [5]. All straight lines 
specified by the user (see their work for more details) are well-preserved, but 
the lines on the floor appear bent, since they are too many to be marked by the 
user. Although their energy minimization formulation guarantees conformality 
and smoothness of the final result, it has the problem of taking some seconds 
to be performed. Our method (Figures]^ and d), does not rely on heavy user 
interaction nor on any optimization and is not restricted to scenes with any 
particular geometry. 

Finally, we compare our approach with the Pannini projection. This projec- 
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(a) Perspective 


(b) Stereographic 




(c) Mercator (cropped at 157°) 


(d) Zorin-Barr, A = 0.2 




(e) MobiuS, 0max = 60° (f) MobiuS, 0max = 120° 

Figure 8: Comparison with standard projections for (j) = 160°. Our method, 
(b), (e) and (f), has a good balance between straight line and object shape 
preservation, obtained by adjusting the parameter (/)max- 
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(c) Mobius, 0max = 60° (d) Mobius, 0max = 110° 


Figure 9: Comparison with recent methods for 0 = 150°. In the result obtained 
by the method in subfigure (a) [24] the different perspective projections used for 
different areas of the image appear clear and unpleasant on the floor of the scene. 
The method in subfigure (b) [5] preserves all straight lines marked by the user, 
but fails to preserve the ones on the floor (which are too many to be marked 
by the user). Note also that marking a large number of lines would impose too 
many constraints on the optimization process and could compromise the final 
result. Subfigures (c) and (d) were produced with our method. In subfigure (c) 
we use the value of 0max we consider perceptually good. Subfigure (d) shows 
how the bending of lines on the floor was slightly corrected by augmenting the 
value of 0 max 5 but without obtaining a perceptually better image. 


16 




























(a) (i = 0 (b) c? = 1 (c) d ^ oo 

Figure 11: Pannini, 0 = 120° 


tion method works as a cylindrical projection, but moving the projection center 
away from the center of the sphere. The projection center c is moved along 
the ray originating at the midpoint o of the cylinder central axis, in a direction 
perpendicular to the projected plane. Assuming the cylinder has radius 1, the 
parameter d represents the distance from o to c. This parameter d controls the 
image compression. If d = 0 (ie., o and c coincide), the resulting projection is 
a central cylindrical projection. If d = 1 (z.e., c is on the surface of the cylinder, 
behind o), the projection is stereographic cylindrical (also called basic Pannini). 
When d ^ oo, the resulting projection is cylindrical orthographic. This way, 
when d grows, points in the borders of the resulting image are not distorted as 
in the perspective projection, producing perceptually pleasant results. Since it 
is a cylindrical projection, Pannini has the property of preserving all vertical 
lines. 

Figures pT] and pT] show the effect of applying our method and the Pannini 
projection, both with different paramenters, to an image of a room with a FOV 
of 120° (the latter produced using an open-source implementation of the Pannini 
method, provided by the Panotools software [7] called through Hugin [6]; please 
note that the Pannini projection was also implemented by its authors as a 
standalone package m)- These two figures show the effect on line bending 
obtained by varying the value of the parameters of each method. We note first 
one of the strengths of the Pannini projection, the preservation of vertical lines. 
For the rest, it can be considered that 0max and d are somewhat analogous. 
We can also observe that big values of d allow for more line bending than small 
values of ^max, but this fact does not mean that images obtained with big values 
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(a) 0max = 40° O.Trad (b) 0max = 1° ^ 0.02rad 

Figure 12: Even if our method can handle very wide FOVs and displayed objects 
are distinguishable, the image seems unrealistic. Here, 0 = 320° « 5.Grad. 


of d are perceptually better. 

To conclude this Section, we recall that our method produced very good 
results for all the tested cases, and was succesfully compared to state-of-the-art 
projection and warping methods. Nevertheless, we should point out that there 
are still some cases for whose our method cannot produce realistic results. For 
instance, Figure \T ^ shows what happens with a very wide FOV, close to 27rrad = 
360°, in a panoramic taken from the lie Saint-Louis in Paris. A perspective 
projection will simply not work in this case. Our method can produce a clear 
but unrealistic image. We believe that this cannot be overhauled, since the 
human vision is unable to observe such a wide FOV. The fact that our method 
handles such a large FOV makes it a candidate for visualization in different 
settings, such as those enumerated in the next Section. 

7 Applications 

The method described above does not depend on the content on the image, 
nor require human interaction to specify which regions of the image should 
be preserved (it is a global projection, not a local warping technique). The 
processing time needed is thus much smaller, making it suitable to be applied 
on panoramic videos and interactive panorama visualizers, such as Google Street 
View [9]. 

In interactive applications, apart from the resulting image, the response 
speed of the interface is very important. One improvement that can be incor¬ 
porated to our technique is to have various input spheres (in practice, various 
equiroctangular images), with different resolutions. This would allow to choose 
one input sphere, based on the current FOV, preventing the projection to pro¬ 
cess unnecessarily large amounts of points. The values of (p for which the sphere 
must be switched must be estimated experimentally. 
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Note that actually something similar to the latter approach is implemented 
in Google Street View where, from one input sphere, multiple projections with 
different resolutions are produced [T]; the viewing sphere is never downloaded 
(this also minimizes the amount of data transmitted, which is another constraint 
to keep in mind when working on a client/server environment). 

Our method can be coupled with common user interfaces of panorama visu- 
alizers. For instance, movements with the mouse when holding a button pressed 
can be used to pan the image (this is, to modify a and A) and the mouse wheel 
can be used to increase and decrease the FOV (that is, varying at the same 
time (pa and (px). These interactions can be alternatively done with keys (for 
instance, panning with cursor arrows while changing the FOV with page up and 
page down keys). 

Since the method is not computationally expensive, considering current tech¬ 
nological developments, it can be implemented by processing transformations on 
a GPU, using a shader. For instance, the image can be a texture to be applied 
on the surface of the viewing sphere. In this setting, when panning, neither 
the sphere nor the texture are altered. When zooming, the texture has to be 
updated in order to reflect the new Mobius transformation applied. We showed 
that applying Mobius transformations in the GPU, using GLSL, is viable and 
yields high frame rates (see Table . 

It is important to stress that, in case of working through a client/server 
environment, the processing can be easily implemented in the client (with local 
GPU computations as described in the previous paragraph), in the server or 
even with a hybrid approach. 

To conclude the present Section, let us mention projection on a dome, an¬ 
other important application of our method. This technique consists in projecting 
an image into the interior surface of a sphere, or a portion of a sphere. Histor¬ 
ically, domes where used in planetariums: they showed the stars exactly as we 
see them. In the last decades, researchers realized that domes could produce 
very pleasant visual experience not only when visualizing stars; a new research 
direction in Computer Graphics born. A dome can provide an immersive envi¬ 
ronment for movies, display of data or games. While specialized hardware tends 
to be expensive, there are modern low-cost approaches which make domes very 
appealing for non-professional uses [HE]- Since the projection surface on a dome 
is not plane, plane projection techniques, which evolved since man started to 
draw maps, are useless. Dome projection methods must consider a sphere, or 
a portion of it, as input, and must project on the surface of the sphere. Points 
on the input sphere must be mapped onto the output sphere. In this setting, 
a Mobius transformation shall be used to directly shrink (or, in this case, also 
expand) the sphere to displace points to the visible part of the sphere, without 
using the last perspective projection step, obtaining thus a plane-project ion-less 
dome visualization. 
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8 Conclusions, Discussion and Future Work 

We introduced a technique based on Mobius transformations of the points on 
the viewing sphere, aimed at improving the perceptual quality of panoramas. 
We validated the new approach by implementing the method, to demonstrate 
both its quality and the speed at which it can run. 

The methods by Zorin and Barr [25], by Carroll et al. [5] or by Wei et al. [23] 
consist in the minimization of energies, which can be used to quantify the percep¬ 
tual quality of the image but, of course, they need human interaction. Milnor’s 
method was presented more than 40 years ago, and it was conceived to help in 
a slightly different problem than ours (map projections). This fact motivates us 
to propose a future research direction: the development of theoretical methods 
aimed at quantifying the quality of projections without user interaction. 

Throughout the paper, we use a constant value to estimate a good value 
for 0max, under the assumption that this value is good on the perceptual side. 
Another interesting research direction would be to study the content of the im¬ 
ages, in order to determine the value of 0max in function of the topology of 
the objects which are present in the image. This research may also provide an¬ 
other approach to measure automatically the perceptual quality of projections. 
Nevertheless, our method can lose the property of being executed in real-time. 
A related possible research direction would be to study the content of differ¬ 
ent frames in a 360° video, in order to dynamically change the projection on 
each frame. This research could also include comparisons with state-of-the-art 
methods for video correction m- 

As mentioned in Section [^ our method can be used to visualize 360° images 
on a dome. This has very important implications that should be discussed here. 
It follows that the well-studied methods that project the viewing sphere onto 
a plane or a cylinder are not useful on this setting. Thus, our method can be 
viewed as a member of another family of transformations on the sphere. We also 
mention that our method can be (and actually was) implemented in real-time, 
what permits to apply it on 360° videos. 

A topic related to dome projection, out of the scope of the paper, which 
can be considered as an application of our method, is zooming images. The 
idea is simple when working on plane images or videos: zooming consists in 
narrowing or widening the FOV. However, this is no longer true in 360° images 
or videos, because the FOV cannot be adjusted. A zoom on a sphere consists in a 
transformation, shrinking some region but, naturally, stretching another region. 
This is exactly what the Mobius transformations presented in this paper do. 
Thus, the transformation of the viewing sphere we proposed throughout the 
paper is a zooming technique for 360° images. 

Finally, let us mention that Mobius transformations are a simple and well- 
known mathematical tool which was not deeply explored in Computer Graphics 
and Image Processing. As mentioned earlier, we believe their applications can 
be extended beyond panorama visualization, namely, where an image is repre¬ 
sented on a viewing sphere. This is an interesting direction to explore in the 
future. To uncover applications of Mobius transformations in visualization, it 
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would be interesting to make a deeper study of the foundations of this method. 
Exploring notions such as the cross-ratio El can shed light to a myriad of new 
techniques for the transformation of images inscribed on the viewing sphere, 
such as changing the perspective of scenes with Mobius transformations. 

It also remains open the question about the perceptual quality of projections: 
is it possible to conceive an analytic method to quantify the quality in a way 
that is consistent with human perception? 
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(a) elliptic 


(b) hyperbolic 


(c) loxodromic 


(d) parabolic 


Figure 13: Four kinds of Mobius transformations (drawings taken from [H]). 

A Mobius Transformations 

We recall in this appendix some properties of Mobius transformations. The 
information here should be enough to understand the paper. We refer to the 
book by Needham [14] for proofs and details. 

A Mobius transformation is a mapping of the form 


M{z) = 


cz d' 


where a, c and d are complex constants. Multiplying these coefficients by 
a constant yields the same mapping, thus what matter are the ratios of the 
coefficients (usually the coefficients are normalized by scaling them to satisfy 
ad — be = 1). This fact shows that only three complex numbers are sufficient 
to determine uniquely a mapping. It can be shown that there exist a unique 
Mobius transformation which sends any three points to any other three points. 

Mobius transformations have fixed points, computed through the equation 
2 : = M{z). Since this equation is quadratic, it has at most two solutions (except 
for the identity mapping). When c 7 ^ 0, both fixed points lie in the finite 
plane. When c = 0, at least one of the fixed points lies at infinity (as it can 
be deduced from from Figure [TSl infinity is represented by the north pole N of 
the Riemann sphere). In this case, the transformation is simplified, taking the 
form M{z) = Az + B. We can write A = in order to view M{z) as the 
composition of a rotation of a centered in the origin, plus an expansion by p and 
a translation of B. This interpretation lets us view geometrically the Mobius 
transformation, as depicted in Figure [TSl 

When a > 0, p = 1 and B = 0, M{z) is a rotation of the complex plane 
(z.e., a rotation of the sphere), as shown if Figure 
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The fixed points of 
this transformation are the two poles of the sphere, which correspond, in the 
complex plane, to the origin and infinity. This is called an elliptic Mobius 
transformation. 

Figure m illustrates the transformation when a = 0, p > 1 and B = 0. 
M{z) is, in this case, an expansion centered in the origin. The two fixed points 
are the same as in the previous case. If a = 0, p < 1 and 5 = 0 , it is an origin- 
centered contraction. These Mobius transformations are called hyperbolic. 
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When a 7 ^ 0, p 7 ^ 1 and 5 = 0, the resulting transformation is a combination 
of the two former cases. This is called a loxodromic Mobius transform and it is 
illustrated in Figure [T^ The two fixed points of this transform are the same 
two as the above cases. 

The last case study is the translation, occurring when A = 0 and 5 7 ^ 0. 
The only fixed point of this transformation is infinity, represented on the sphere 
by the north pole. This transformation, the parabolic Mobius transformation, 
is depicted in Figure p^d| 
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